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ABSTRACT.  An  extremum  principle  is  dexelopcd  that  determines  three-dimensional  surface  orientation 
from  a  two-dimensional  contour.  I'he  principle  maximizes  the  ratio  of  die  area  to  the  sqm-ire  of  the 
perimeter,  a  measure  of  die  compactness  or  symmetry  of  the  three-dimensional  surface.  Ihe  principle 
interprets  regular  figures  correctly  and  it  in  .iprcts  skew  symmetries  as  oriented  real  symmetries.  The 
maximum  likelihood  method  approximates  die  principle  on  irregular  figures,  but  we  show  that  it  consistently 
overestimates  die  slant  of  an  ellipse. 
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1.  introduction 

An  important  goal  of  carl>  vision  is  the  compuliilion  of  a  representation  of  the 
visible  surfitces  in  an  image,  in  particular  the  detertnination  of  die  orientation  of 
those  surfaces  as  defined  by  their  local  surface  normals  |Marr  1982.  Hradv  1982], 
Many  prcKesses  contribute  to  achieving  this  goal,  stereopsis  and  structure  from 
motion  being  currently  tire  most  studied  in  image  understanding.  I  hrec  other 
impevrtant  contributing  processes  are  shape- froin-contour.  shape-from-texture-- 
gradients.  and  shape-froni-shading.  Several  psychophysical  demonstrations  show 
that  shape-from-contour  is  signiHcanily  more  powerful  tlian  shape-from-texturc- 
gradients  (Clark  et.  al.  1956.  Gruber  and  Clark  1956.  Hraunstein  and  Payne, 
1969J.  Similarly,  Barrow  and  Tenenbauin  (1981,  Figure  1..1  ff(  suggest  that 
shape-from-contour  is  a  more  cfTective  clue  to  shape  than  shape- from-shading. 

In  this  paper  we  consider  the  computation  of  shape-from-contour.  Figure  1 
shows  a  number  of  shapes  that  are  typically  perceived  as  images  of  surfaces 
which  arc  oriented  out  of  the  picture  plane.  I'hc  corresponding  (sets  of)  surface 
normals  (up  to  tilt  reversal)  are  shown  to  die  right.  It  may  be  supposed  (sec 
for  example  (Gregory  1973.  pages  168ft])  that  the  slant  judgements  in  F'igure 
1  arc  largely  determined  by  familiarity  w  ith  regular  shapes  such  as  circles  and 
squares.  Ftgure  2  strains  that  hypodicsis  (though  Gregory  proposes  diat  we  arc 
familiar  with  die  shape  of  puddles)  and  suggests  that  die  conipuUition  is  based 
on  more  general  knt'wledgc  of  shapes  and  surfaces.  The  mediod  we  propose 
is  based  on  such  general  knowiedge.  namely  a  preference  for  symmetric,  or  at 
least  compact,  surfaces.  Note  that  the  conU)ur  docs  not  need  to  be  closed  in 
order  to  be  interpreted  as  oriented  out  of  the  image  plane.  Finally.  Figure 
3  shows  that,  in  general,  contours  arc  interpreted  as  curved  three-dimensional 
surfaces. 

We  develop  an  extremum  principle  for  determining  three-dimensional  surface 
orientation  from  a  two-dimensional  contour.  Initially,  we  work  out  the 
extremum  principle  for  the  cases  illustrated  in  Figures  1  and  2,  that  is,  assuming 
0  priori  that  the  contour  is  closed  and  that  the  interpreted  surface  is  planar. 
I.atcr,  we  discuss  how  to  extend  our  approach  to  open  contours  and  how  to 
interpret  contours  as  curved  surfaces  as  shown  in  Figure  3. 

The  extremum  principle  maximizes  a  familiar  measure  of  the  compactness  or 
symmetry  of  an  oriented  surface,  namely  tlie  ratio  of  the  area  to  the  square  of 
the  perimeter.  It  is  shown  that  this  measure  is  at  the  heart  of  the  maximum 
likelihood  approath  to  shape-from-contour  developed  by  Witkin  (1981)  and 
Davis.  Janos,  and  Dunn  (1982).  I'lie  maxiimiiii  likelihood  approach  has  had 
some  success  interpreting  irregularly  shaped  objects.  I'hc  method  is  incircctive. 
however,  when  the  distribution  of  image  tangents  is  not  random,  as  is  the 
c.ise.  for  example,  when  the  image  is  a  regular  shape,  such  as  an  ellipse  or 
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Figure  1 

Two-dimensional  contours  that  are  often  interpreted  as  planes  that  are  oriented  with 
respect  to  the  image  plane.  The  commonly  judged  slant  is  shown  next  to  each  shape. 


Figure  2 

Some  unfamiliar  shapes  that  are  also  interpreted  as  planes  that  are  oriented  with  respect 
to  the  image  plane.  Ihe  shape  on  the  left  is  Iruni  |(hegor)  197.T  fig.  lO.V],  tlte  others 
from  (Wilkin  1980.  p  29  and  94]. 


a  par.illehrgi.ini.  Ottr  cxtrciiiiiiii  principle  iiitoiprcts  rcctilar  liguics  correctly. 
We  show  that  the  m.ixmiiiiii  likelihood  ineihod  appioxim.ites  the  extreiniini 
principle  for  irrectil.tr  fiptires;  bm  that  the  in.ixitmim  likelihood  method  docs 
not  compute  the  correct  slant  lor  .in  ellipse.  Uilkin  (I9SI.  I  ietiie  provides 


Figure  3 

Some  shapes  that  are  interpreted  as  curved  three-dimensional  surfaces. 

empirical  evidence  Uiai  the  maximum  likelihood  method  computes  a  good 
approximation  to  the  perceived  lilt  but  undcrcsiitmie^  the  slant.  We  prove  in 
Appendix  A  that  the  maximum  likelihood  method  consistently  mercsiinuUes  the 
slant  of  an  ellipse.  A  more  thorough  investigation  of  the  difference  between 
the  Hxtrcinum  Principle  and  the  Maximum  Likelihood  method  is  needed. 

One  class  of  figures  that  arc  readily  perceived  as  lying  in  a  plane  other  than 
the  image  plane  arc  skew  symmetries,  which  arc  two-dimensional  linear  (affine) 
transfonnations  of  real  symmetries.  Kanadc  (1981.  page  424)  has  suggested  a 
method  for  determining  the  three-dimensional  orientation  of  skew-symmetric 
figures,  under  the  "heuristic  assumption"  that  such  figures  arc  interpreted  as 
oriented  real  symmetries.  We  prove  that  our  extremum  principle  necessarily 
interprets  skew  symmetries  as  oriented  real  symmetries,  thus  dispensing  with 
the  need  for  any  heuristic  assumption  to  that  effect.  Kanadc  shows  that  there 
is  a  one-parameter  family  of  possible  orientations  of  a  skew-symmetric  figure, 
forming  a  hyperbola  in  gradient  space.  He  suggests  that  the  minimum  slant 
member  of  tlic  onc-parainelcr  family  is  perceived  In  the  special  case  of  a  real 
symmetry,  Kanadc's  suggestion  implies  that  symmetric  shapes  arc  perceived  as 
lying  in  the  image  plane,  that  is  having  zero  slant.  It  is  clear  from  the  ellipse 
in  figure  1  that  this  is  nut  ctirrccl.  Our  mctliod  interprets  real  symmetries 
correctly. 

first,  we  review  the  m.iximum  likelihtMid  method.  In  Section  3.  we  discuss 
several  previous  cxtrcmiitn  principles  and  justify  our  choice  of  tlie  compactness 
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mcMsiiic.  In  Scciiiin  4,  »<c  derive  tlie  m.illieiiuilics  iicccssaii  U)  cxiicmi/c 
the  (.ump.ieliK'ss  inc.isiiic,  and  relate  the  cxirenmm  pnntiple  Id  llie  ni.ivimniii 
likelihood  method.  In  Section  5.  v\c  iinesiig.ite  K, made's  woik  on  skew 
svmmetrv.  One  approach  to  extending  the  extremum  principle  to  inierinet 
curved  surfaces,  such  as  that  shown  in  Figure  is  sketched  in  Section  6.  In 
the  linal  section,  we  relate  tltis  work  to  die  psychophysical  literature  on  slant 
estimation  and  image  understanding  work  on  shape  from  texture. 

2.  The  .Sampling  Approach 

Witkin  |1981]  has  treated  the  determination  of  shapc-from-contour  as  a  problem 
of  signal  detection.  Recently.  Davis,  Janos,  and  Dunn  (I982J  have  corrected 
some  of  Witkin's  mathematics  and  proposed  two  cITicicni  algorithms  to  compute 
the  orientation  of  a  planar  surface  from  an  image  contour.  Witkin’s  approach 
uses  a  ^eoim'inc  moilrl  of  (orthographic)  projection  and  a  shuisiical  model  of 
(a)  tlie  distribution  of  surfaces  in  space  (suiiistics  of  the  universe)  and  (b)  of 
the  distribution  of  tangents  to  die  image  contour.  We  shall  adopt  the  geometric 
model,  but  dispense  with  tlie  statistical  model  in  favor  of  an  extremum  principle. 

f  irst,  the  geometric  model.  Assume  that  tlie  image  plane  is  hori/ontal  with 
coordinates  (i,y)  (sec  figure  4)  .  lo  obtain  a  plane  with  slant  a  and  tilt 
r,  we  rotate  (x,y)  by  r  in  the  image  plane  and  then  rotate  the  image  plane 
by  a  about  the  new  y  axis.  Assume  that  the  coordinate  frame  in  the  plane 
{o,t)  is  chosen  so  that  it  projects  into  (i,  j/).  (In  Section  4.  we  describe  this 
Iransfomiation  more  precisely,  sec  also  Davis.  Janos,  and  Dunn  (1982,  p3).) 

Now  suppose  that  a  curve  is  drawn  in  the  plane  (o,t)  and  denote  by  0  the 
angle  tliat  the  tangent  makes  at  a  typical  point  on  the  etirve.  I.et  a  be  the 
tangent  angle  in  tlie  image  plane  at  the  point  corresponding  to  0.  I’hcn  a  and 
0  arc  related  by: 


tan{Q  —  r)  =  (2.1) 

cos  a 

We  now  turn  to  the  statistical  model,  which  consists  of  two  assumptions 
called  isotropy  and  independence.  Isotropy  reasonably  supposes  that  all  surface 
oncnt.itions  arc  equally  likely  to  occur  in  nature  and  that  tangents  to  surface 
curves  .ire  equally  likely  in  all  directions.  More  succinctly  ,  it  is  assumed  that  tlie 
quantities  (0,a,T)  arc  randomly  distributed  and  their  joint  probability  density 
function  ("density”)  D{0,a,T)  is  given  by  (Davis.  Janos,  and  Dunn.  1982]: 

I){0,o,t)  sin  a  (2.2) 

Wc  assume  ih.it  the  ranges  of  the  angles  arc;  0<c<  §,0<T<7r,0< 
0  <  71,  Simil.ivly,  the  density  of  (a,T)  is  given  by  ; 
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surface 


Figuie  4 

The  geometry  of  orthogniphic  projection.  See  text  for  details  of  notation. 


D(cr,r)  =r  isin  ff  (2.3) 

TT 

llic  independence  assumption  requires  dial  the  image  tangents  {a,:l  <  t  <  n) 
are  statistically  independent.  I'hat  is.  it  is  assumed  that  tlie  tangent  directions 
at  diirercnt  points  on  the  image  curve  are  independent.  Iliis  is  only  true  if 
tlie  contour  is  highly  irregularly  shaped,  or  if  the  number  ttf  samples  is  small. 
In  any  case,  the  assumption  of  independenee  is  an  inherent  weakness  of  the 
sampling  approach  (see  fur  example  (Witkin  1981.  p.  36]). 

Witkin  (1981,  p.  25  -  26]  shows  that  the  joint  density  ftinction  D(Q,a,r)  is 
given  by: 
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sin  a  COB  a 


D(a,a,T}=  ^ 

cos^(a  —  t) -f- cos*asin*(a  —  r) 
For  the  conditional  density  D{a\(j,T)  we  find: 


(2.4) 


D{Q\a,  t)  =  - - - - - .  (2.5) 

cos^(q  —  r)  +  cos^  aBin*(a  —  r) 

Denote  the  s,implc  (01,02, ...o„)  by  A  (the  sample  is  independent  by 
assumption).  It  h.is  conditional  density 
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D(A\a,T)  =  I)[q,\o,t)  (2.6) 

1^1 

Hy  Bayes’  fomuila  we  obtain 


D{a,r\A)  = 


/  /  D{A\o,  T)J)(a,  T)dodT 


(2.7) 


Observe  that  die  numerator  is  independent  of  a  and  t.  The  sampling  approach 
takes  a  random  sample  A  and  defines  the  most  likely  orientation  of  the  plane 
(o,r)  to  be  that  which  cxtremi/cs  D{o,t\A).  Witkin  [1981]  quantizes  a  and 
r,  and  describes  an  algoridim  to  find  the  maximizing  (o^, tj.).  Oavis,  Janos, 
and  Dunn  [1982]  dciclop  a  more  eflicicnt  algorithm  that  first  estimates  a  and 
T  and  then  uses  diosc  estimates  in  a  Newton  iterative  prtKcss.  They  provide 
evidence  that  their  method  is  more  accurate  dian  Witkin’s.  Curiously,  however, 
they  state  [Davis,  Janos,  and  Dunn  1982,  p  24]  that  "the  iterative  algoridim  was 
not  used  [in  the  experiments  they  report)  because  the  iniiiial  estimates  (whose 
compuution  is  trivial)  arc  very  accurate  and  die  iterative  scheme  often  failed 
to  converge  to  the  solution". 


3.  Kxtremuin  Principles. 


Brady  and  Horn  [1983[  survey  the  use  of  extremum  principles  in  image 
understanding.  1  he  choice  of  performance  index  or  mcastirc  to  be  extremiz.ed. 
and  die  class  of  functions  over  which  die  extrcmizaiion  takes  place,  arc  justified 
bv  .(ppcalitig  to  a  model  of  die  geometry  or  photometrv  of  image  forming  and 
constniiiits  such  as  smoothness.  Tor  example,  die  use  of  cxtrcmimi  principles 
in  surface  reconstruction  is  based  upon  surface  consistency  dieorcms  [Crimson 
1981.  1982,  198.1,  Vudle  1983)  and  a  thin  plate  model  of  visual  surfaces  [Brady 
and  Horn  1983.  Tcrzopoulos  1983). 

Ilicrc  arc  several  plausible  measures  of  a  curve  that  might  be  extremized  in 
order  to  compute  shape-froin-coiitour.  Tirst.  §  K^ds.  where  k,  is  the  curvature 
of  the  contour,  h.is  been  investigated  as  a  curve  of  least  cnergv  for  interpolating 
across  gaps  in  plane  curves  [Horn  1981).  Here  we  seek  a  measure  of  a  curve 
di.it  is  extremized  when  the  pi. me  containing  the  curve  is  slanted  and  tilted 
appro|iri.itcly.  Contrary  to  what  appears  to  be  a  popul.ir  belief,  given  an  ellipse 
in  the  im.ipc  pl.me.  /  s"d,s  is  iwi  extremized  in  the  pi, me  tli.it  tr.msfomis  the 
ellipse  into  .1  circle.  .Appendix  B  contains  ,i  proof  of  this  assertion.  Since 
ellipses  .ire  norm, illy  perceived  .is  slanted  circles,  we  reject  the  squ.ire  curvature 
.IS  .1  suit.ible  measure. 

.\notlier  possible  measure  is  proposed  by  Barrow  and  Teneiih.iimi  [1981,  p89]. 
.\ssmmiig  pl.m.iiity  (the  torsion  r  is  zero),  it  reduces  to 
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One  ohjcctinn  lo  this  mc.isurc  is  th.it  it  involves  hipli-ordcr  dcruatives  of  the 
curve.  This  means  it  is  overly  dependent  on  sin.ill  st.ilc  bcli.oiour.  Consider, 
for  example,  a  curve  which  is  circular  except  for  a  small  kink.  Ihc  circular 
part  of  the  curve  will  contrihutc  a  tiny  proportion  to  the  intecral  even  when 
the  plane  containing  the  curve  is  rotated.  I  he  kink,  on  the  other  hand,  will 
contrihutc  an  arhitrarily  large  proportion  and  so  will  dominate  the  integral  no 
matter  how  small  it  is  compared  with  the  rest  of  the  curve.  This  is  clearly 
undesirable.  Tor  cx;iniple.  it  suggests  that  tlic  measure  will  be  highly  sensitive 
to  noise  in  tlte  position  and  orientation  of  die  points  forming  die  contour. 

A  second  objection  to  the  measure  proposed  by  Harrow  and  Tcnenbaiim  is 
that  it  is  minimized  by.  and  hence  has  an  intrinsic  preference  for.  straight  lines, 
for  which  dulds  zero.  This  means  that  the  measure  h.as  a  bias  towards  planes 
that  correspond  to  the  (non-general)  side-on  viewing  position.  These  planes  are 
perpendicular  to  the  image  plane  and  have  slant  7r/2. 

We  base  our  choice  of  measure  on  the  following  observations. 

1.  Contours  that  are  the  projection  of  curves  in  planes  with 
large  slant  arc  most  cttectivc  for  eliciting  a  dncc-dimensional 
intcipretation. 

2.  A  curve  is  foreshorted  by  projection  by  the  cosine  of  the  slant 
angle  in  the  lilt  direction,  and  not  at  all  in  the  orthogonal  direction. 

We  conclude  that  three-dimensional  interpretations  .arc  most  readily  elicited 
for  shapes  diat  are  highly  elongated  in  one  direction.  Another  way  lo  express 
this  idea  is  that  die  image  contour  has  large  aspect  ratio  or  is  radially  asymmetric. 
The  measure  we  suggest  will  pick  out  the  plane  orientation  for  which  the  curve 
is  most  compact  and  most  radially  symmetric.  Specifically  our  measure  is 


M  = 


(Area) 

[PerimeterY  ’ 


(3.1) 


This  is  a  scale  invariant  number  characterizing  the  curve.  For  all  possible  curves 
it  is  maximized  by  the  most  compact  one.  a  circle.  IJy  compact,  we  mean  most 
radially  symmetric.  This  gives  the  measure  an  upper  bound  of  l/47r.  Its  lower 
bound  is  dearly  zero  and  it  is  achieved  for  a  straight  line.  It  follows  that  our 
measure  has  a  built-in  prejudice  against  side-on  views  for  which  the  slant  is 
rr/2. 

In  general,  given  a  contour,  our  extremum  principle  will  choose  the  orientation 
in  which  the  dcprojected  contour  maximizes  M .  For  example  an  ellipse  is 
interpreted  as  a  slanted  circle.  The  till  angle  is  given  by  die  minor  axis  of  the 
ellipse.  It  is  also  Mr.iightforward  to  show  that  a  parallelogram  corresponds  to  a 
rotated  sgu.ire.  Appendix  C  discusses  die  interpretation  of  several  sitiipic  shapes. 
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In  particular,  an  ellipse  is  interpreted  as  a  slanted  circle,  a  parallelogram  as  a 
slanted  square,  and  a  triangle  as  a  slanted  ei|uil.iteral  triangle.  In  Section  5  v\e 
extend  the  parallelogiam  result  to  the  more  general  case  of  skewed  symmetry. 

\Vc  note  that  the  quantity  A/  is  commonly  used  in  pattern  recognition  and 
industrial  vision  systems  [Agin  1980,  Pavlidis  1977.  Mallard  and  Mrown  1982] 
as  a  feature  that  measures  tlic  compactness  of  an  object.  Furthermore,  we  can 
show  that  the  measure  M  defined  in  I  .q.  (.VI)  is  at  the  heart  of  llic  geometric 
model  in  the  maximum  likclihoud  approach. 

From  Section  2.  we  sec  that  tlie  maximum  likelihood  approach  maximizes 
the  product  of  a  number  of  terms  of  form 


//  \  cos  a 

f(a)  =  — - - - - .  (3.2) 

cos'(a  —  r)  +  cos*  osin^{Q  —  t) 

Differentiating  Fx^.  2.1  with  respect  to  the  arc  length  s;  along  the  image  curve 
and  Sfi  along  the  routted  curve  respectively  we  obtain 


Kl  dS]  1 

rfsR  f(ot) 


(3.3) 


where  /c;  and  kr  arc  Utc  curvature  at  corresponding  points  of  the  image  contour 
and  its  dcprojcction  in  the  rotated  plane  respectively.  In  fact,  k./  —  dalds]  and 
Kfi  =  dp/dsR.  I'hcrc  is  no  a  or  t  dependence  in  the  numerator  of  equation 
(3..1).  We  can  write  each  term  Kds  as  {dsds)/{pds)  where  p  is  the  radius  of 
curvature.  Now  observe  that  (pds)/{ds  ds)  is  just  a  local  compuUition  of  area 
divided  by  perimeter  squared!  FIcnee  maximizing  each  /(a)  in  die  maximum 
likelihood  approach  is  equivalent  to  locally  maximizing  area  over  perimeter 
squared.  In  section  (4)  we  will  examine  this  connection  more  rigorously. 

Finally  we  note  that  the  area,  as  well  as  the  perimeter,  can  be  obtained  by 
an  integral  round  tlic  contour.  If  n  is  the  normal  to  the  curve  then  it  is  a 
straightfttrward  applictition  of  Stokes'  nicorcm  to  show  tJiat 


(Area)n  =  r  X  dr,  (3.5) 

where  (Area)  is  a  scalar  quantity,  and  r  is  a  vector  cotudinate  system  in  the 
pl.inc  of  the  figure.  I  his  formula  simplifies  tlic  calculations  and  means  the 
perimeter  and  the  area  can  be  computed  simultaneously. 


4.  Fxtreini/ing  the  Measure 


We  now  write  down  the  measure  for  a  curve  with  arbitr.iry  oricnt.ition  and  then 
extremi/e  with  respect  to  the  orientation.  I  et  the  tinit  noimals  to  the  image 
plane  .ind  the  rotated  plane  be  k  and  n  respectively,  the  slant  o  of  the  rotated 
plane  is  given  by  the  scalar  product 
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cos£7  =  k  •  n  (4.1). 

Let  P/i  and  Py  he  tlie  contour  in  the  rotated  and  image  planes.  A  \ector  r  in 
the  image  plane  siitislics  r  •  k  =  0,  and  is  the  projection  of  a  vector  v  in  the 
rotated  plane  iliat  s;itisfies  ♦  •  n  =  0.  The  projection  relationship  between  v  and 
its  image  r  is  defined  b>; 

r  =  k  X  (v  X  k)  =  V  —  (v  •  k)k  (4.2) 

_  nx  (rx  k)  (n-k)r-(n-r)k  ,, 

'  -  -(iao-  =  (n.k) 

where  X  denotes  vector  product  Now  V ft  and  Py  have  (vector)  areas  Ay?  and 
Ay  given  by 


Ay?  =  ^  /  V  X  dv  (4.4) 

A/  =  ^  ^  r  X  dr  (4.5) 

Observe  that  the  area  vectors  have  the  same  direction  as  tlic  normal  to  the  plane 
containing  the  area.  In  particular.  Ay?  is  in  direction  n  and  Ay  is  in  direction 
k.  Substituting  Hq.  4.3  into  Kq.  4.4  and  using  Hq.  4.1,  we  find 


It  follows  that 


Ay?  =  Ay  -f 


k  X  (n  X  Ay) 
(n-k) 


COS  a 


llAflll  = 


IIA/II 

cos  a 


(We  recall  that  the  range  of  a  guarantees  that  cos  o  is  positive. 
1  he  perimeter  lengths  Fy?  and  Pj  arc  given  by 


(4.6) 


(4.7) 


Fy?  =  /^J|dv|| 

(4.8) 

Pi  =  lm 

(4.9) 

Substituting  Hq.  4.3  into  Hq.  4.8  gives 

(4.10) 

In  gcncnil  tiicre  is  no  simple  relationship  between  the  perimeters  analogous  to 
l,q.  4.6  between  the  areas.  Nesenheless.  by  l-lq.  4.7. 


P\  P%  cos  a 


(4.11) 


and  so  our  extrentvim  principle  is  equivalent  to  extremi/ing  cosi  aPn,  which 
we  wiite  as 


+  (4., 2) 

We  extremi/c  this  with  respect  to  the  orientation  n  of  the  rotated  plane, 
maintaining  tlie  constraint  that  n  is  a  unit  vector  by  a  Lagrange  multiplier  A/4. 
This  gives 

ir)^\  ^  (n  •  k)^(ir^  —  (n  ■  dr)^ 

Wj  FkP  “ 

(n  (fr)^'\  ^  (n  ■  k)(n  •  £fr)rfr  (^-^3) 

(n  k)  J  Tn  kj2 

+  An  =  0 

faking  scalar  products  of  };q  (4.13)  with  k  and  n.  respectircly.  gives 


kj  ^(n  .  k)dr^ -f 
+  2j^ink 


k)(fr  4" 


0  =  (n  •  k)A 


(4.14) 


(4.15) 


Wc  use  Fq.  4.14  to  remove  the  integral  coefficient  of  k  in  F.q.  4.13.  allowing 
us  to  express  Fq.  4.13  as  a  sum  of  tJie  second  integral  and  k  X  (k  X  n)  times 
A.  Wc  now  use  Fq.  4.15  to  eliminate  A  from  this  new  form  of  Fq.  4.13.  Wc 
recall  that  the  unit  tangent  t  is  defined  by 


dr 

Ts’ 


where  .s  is  are  length  along  the  contour.  It  follows  that 


II 


kcciilling  ili.u  (ii  ■  k)  -  :  rose,  we  find 

2y^|cos'‘*£r4- (n  ()^|  ^(t  ii)</r=  -k  X  (k  X  n)  f  jeos^  o  +  (n  ■  l)"^} 'rfs 

(4.16) 

where  t  =  <ir/||dr||  is  the  unit  tangent  to  tlic  image  contour. 

l  ot  the  unit  \eetors  in  the  x  and  y  dircTtions  in  the  image  plane  be  i  and  j 
and  the  normal  to  the  image  plane  k.  By  definition,  costr  =  k  ■  k.  Ihe  tilt  t 
is  defined  by 


The  tangent  vector  t  and  the  normal  n  can  be  written: 


t  =  cos  ai -|- sin  oj  (419) 

n  =  sin  a  cos  ri  -f  sin  o  sin  rj  -f  coserk  (4-20) 

where  a  is  the  tangent  angle  in  the  image.  We  now  form  die  scalar  products 
of  l-.q.  4.16  with  i  and  j  to  obtain  (with  appropriate  cancelling) 


2  y"  (cos^  a  +  sin^  (7cos*(a  —  t))  i  cos(a  —  t)  cos  ads 
=  COST  ^  (cos^o  -f  sin^acos*(Q  —  T))^ds 


(4.21) 


2  ^(cos^  a  -|-  sin"  (7cos^(a  —  t))  i  cos(q  —  r)  sin  ads 
=  sluT ^(cos^a  -(-  sin^  <7cos*(q  —  T))i(is 


(4.22) 


we  can  rewrite  these  equations,  after  multiplying  by  factors  of  :±  cos  r  and 
±  sin  r.  as: 

2  ^  (cos^  o  -f  sin^acos^(Q  —  r))  co8*(cr  —  T)ds 

=  j  (cosher  +  sin*(7cos*(Q  —  T))^ds  (4.23) 

2  J^(cos^o  +  sin^ocos*(a  —  T))~i  cos(a  ~  T)sin(Q  —  rjds 


(4.24) 
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Wc  use  l-qs.  4.2.1  and  4.24  in  Appendix  I)  to  dcierminc  tlie  cxtrcnii/ing 
orientation  for  a  skew  symmetry.  We  can  rewiiic  tJiesc  equations  in  the  form: 


da 

dr 


{A-h 

'cos^  a 

(-r/' 

'cos5  a 


cos*  a  4"  sin*  t7cos*(a  —  t 
cos*  a  -{-  sin*  acos*(Q  —  r 


))^dsj  =  0  (4.25) 

))i(fsj  =  0  (4.26) 


to  emphasize  that  they  correspond  to  extremizing  with  respect  to  a  and  t. 

Wc  can  implement  the  extremum  method  directly  from  Kqns.  (4.25)  and 
(4.26)  by  quantizing  the  tangent  angle  q„  slant  o^.  and  tilt  Tk.  replacing  the 
integral  by  a  sum.  Iliis  gives  good  results,  e\en  though  we  use  fixed  point 
integers  in  our  edge  finder.  A  multi-level  search  speeds  the  algoritlim  by  a 
factor  of  ten.  For  large  slant  the  ratio  of  the  greatest  to  least  value  of  the 
expression  is  large,  and  the  result  is  numerically  well-conditioned.  For  smaller 
slants  (less  than  about  45  degrees  in  the  case  of  an  ellipse)  die  ratio  is  small 
and  the  result  poorly  conditioned,  so  that  round-off  errors  can  be  significant. 

l  o  conclude  this  Section,  wc  show  that  these  equations  arc  similar,  though  not 
identical,  to  those  obtained  by  the  maximum  likelihood  method  in  the  limit  as 
the  number  of  sampled  uingcnts  tends  to  infinity.  I  o  see  this  wc  recall  from  Kq. 
2.7  that  Uiis  method  invokes  extremizing  0{A\a,r)  with  respect  to  a  and  r. 
Since  the  denominator  is  independent  of  a  and  r.  this  amounts  to  extremizing 
D{A\a,  T}D{a,T).  ITiis  is  the  same  as  extremizing  log  D(A\a,  T)D(a,  t).  Using 
2.3,  2.5  and  2.6  wc  obtain: 


E  =  n  log  COSO  +  log  sin  o  —  ^  log(cos*(Q,  —  r)  -f-  cos*  (Tsin*(a,  —  t)), 

(4.27) 

where  wc  have  ignored  factors  of  ir  which  will  vanish  on  differentiation. 
Dividing  E  by  n  and  diking  the  limit  as  n  tends  to  infinity  gives: 

F  =  log  cos  CT  ^  dr  —  ^  log(cos*(Q  —  r)  -|-  cos*asin*(Q  —  r))(ir.  (4.28) 
Using  die  identity; 


cos*(q  —  t)  f  cos*(7sin*(Q  —  t)  =  cos*o  +  sin*(7cos*(a  —  t)  (4.29) 
gives 

F  =  log  ease  f  ^  log(cos*  a  -f-  sin*  ocos*(q  —  T))dr.  (4.30) 
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I  his  (oirtiiil.i  is  similar  tc  I  qs  4.25  and  4.26.  I  hus  cspsxi  the  l-AUvnuim 
Molhod  1(1  iihs’  simil.ir  rcsiilis  lo  the  .Sainplint;  Mciliod  when  ihc  ccnuuir  is 
siiHisiciul)  iiiciiiilai.  tliittcvoi.  we  tan  sluiH  l()iinall\  ill. il  Ihe  Sampling  Meibod 
and  Iho  I  xiieiiuiin  Method  </«■»;«/ eqiiisaicnt.  In  Appendix  A  we  show  lh.it  ilic 
Sampling  Method  oxcresiim.iies  the  slant  of  an  ellipse.  Hie  preeise  distrcpancy 
between  llie  methods,  .iiid  its  praetical  consequenee  for  eonipiitmg  shape  from 
conluur  is  entrcnlK  under  inxestigation. 

5.  Skew  Syiimietry 

Wc  now  consider  a  more  genera!  class  of  shapes  for  which  the  maximum 
likelihood  approach  is  not  effective.  Kanadc  [1981.  sec.  6.2]  has  introduced 
skew  symniciries.  which  arc  two-dimensional  linear  l.illinc)  ir.insfonnations  of 
rc.il  symmetries,  fhere  is  a  bijectixe  correspondence  between  skew  symmetries 
and  images  of  symmetric  sh.ipes  that  lie  in  planes  oriented  to  the  image  plane. 
Kanadc  proposes  tire  heuristic  assumption  that  a  skew  symmetry  is  interpreted 
as  an  oriented  real  sy  mmetry,  and  he  considers  the  problem  of  computing  the 
slant  and  tilt  of  the  oriented  plane. 

Denote  the  angles  between  the  i-axis  of  the  image  and  tire  images  of  the 
symmetry  axis  and  an  axis  orthogonal  to  it  fthe  skewed  transverse  axis)  by  a  and 
P  respectively,  l  ire  orthogonality  of  the  symmetry  and  transverse  axes  enable 
one  cimstraint  on  the  orientation  of  the  plane  to  be  derived.  Kanadc  uses 
gradient  space  (p,  (?)  (Horn.  1977,  llrady.  1982]  to  represent  surface  orientations. 
He  shows  (Kanadc  1981.  p.  425]  that  the  heuristic  assumption  is  equivalent  to 
requiring  the  gradient  [p,q)  of  the  oriented  plane  to  lie  on  tlic  hyperbola 

pjeos^  _  qrjsin*  =  -  cos(a  —  0)  (5.1) 

where 

Pi 
9i 

Kanadc  (1981.  p.  426|  further  proposes  th.at  the  vertices  of  the  hyperbola, 
which  correspond  to  the  least  slanted  orientation,  arc  chosen  within  this  onc- 
paramcicr  tamilv.  I  his  proposal  is  in  accord.incc  with  a  heuristic  observation  of 
Stevens  [I98f)|,  In  the  special  case  that  the  skew  symmetry  is  a  real  symmetry, 
tliai  is  in  the  case  that  n  -  3  =  the  hypcrbolii  reduces  to  a  pair  of 

orthogonal  lines  (Kan.idc  1981.  page  426[  p.issing  through  the  origin.  In  such 
cases  the  slant  is  /cro.  In  other  words,  Kanadc's  propos.il  predicts  that  real 
svninuiiics  ,iiv  mevii.ilily  mieipicicd  .is  lying  m  the  im  ige  pl.iiic,  .ind  hence 


"  anahsk  r. 

''■'Wmctrtf)  "'■  '“PMiiiin  „r,  , 

cm  '"'^'■pretcd^  ’’’  for  k-  he  r 

in  'f>  /iici  hA  /  °  <'■'  real  c,,  ^-inadc'^  ‘^‘^'■''ect 

portion  P, 

A,  ^  o.. 


(5.3) 
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r 


and 


I\  =  2^/-^  +  (6  -  aj^. 

Kroin  l  igiirc  6b  wc  find 


and 


A[  —  '2al  cos  a  +  (6  —  a)/  cos  q 

=  At  COSQ 


(5.-1) 


(5.5) 


P[  =  \JPcos‘^  Q  -f  (6  —  a  +  /  sin  q)^  +  ^ Pcos  -f  (6  —  a  —  /  sin  q)^. 

(5.6) 


Wc  need  to  show  dial 


cos  a  1 

Ibis  is  cqui\alent  to  showing 


(5.7) 


(2  cos  Q  —  1  )(/^  -f  (6  —  tt)^) 

<  4-  (b  —  af  4-  2(6  —  a)/  sin  q|  |/^  4"  (^  —  “)^  —  2(6  —  a)/  sin  q|  . 

(5.8) 

This  condition  clearly  holds  for  7r/2  >  a  >  tt/S.  Assume  therefore  that 
0  <  a  <  7r/3.  Squaring  both  sides  of  (5.8),  we  sec  that  the  condition  is 
equivalent  to 

4-  (6  -  a)'  4-  /"(6  -  0)^(2  4-  (5-9) 

COS^  Q  —  cos  Of 

On  completing  the  square  wc  sec  this  always  holds  provided 


-IcosQ  —  1  —  3cos^  Q  >  0  (5.10) 

Ibis  is  so  provided  ^  <  cosq  <  1.  that  is  for  0  <  q  <  j.  It  follows  that  the 
ratio  of  the  area  to  the  perimeter  squared  is  maximized  for  each  region  when 
Q  =  0. 

Wc  now  partition  the  shape  into  n  regions  as  shown  in  Figure  5.  l.et  the 
block  have  area  A[  and  perimeter  P[  when  tlic  skew  angle  is  q.  and  denote 
the  area  by  A,  and  the  perimeter  by  P,  when  q  is  zero.  It  follows  from  the 
above  results  tliat 

A[  =  Ai  cosa  (511) 


t 
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(b) 


Figure  6 

a.  A  basic  region  before  skewing  it.  b.  Fhe  result  of  skewing  the  region  shown  in  (a) 
through  angle  a. 


and 


A  ^  A 

P'*  ^  P,*' 

(5.12) 

W'c  conclude  from  Fqs.  (5.11)  and  (5.12)  that 

P'*  >  Pj^COBQ. 

(5.13) 

Hence  we  obtain 

^2  P?  >  cos  a  p2 

»=i  t=i 

(5.14) 

n  n 

K  =  cos  a  A,. 

(5.15) 

1=1  1=1 

It  follows  that 

}:r  1^:  .  >:r=,A 

LT=.P;^ 

(5.16) 

and  i.iking  (he  limit 

as  n  tends  to  infinity  shows  that  our 

extremum  principle 

intcrpieis  a  skew  svmmeiry  as  a  oriented  real  s>mmctry. 
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6.  Strategy  and  Sensiliyitv 

In  Section  3,  we  sugcesied  iliat  railiailv  asymmetric,  eloncaicd.  or  non-compact, 
contours  most  readih  elicit  three-dimensional  interpretations.  We  then  proposed 
a  compactness  me.isure  and  suggested  that  perceived  siirracc  orientation 

corresponds  to  the  slant  and  tilt  dial  maximi/e  the  measure.  We  hriclK  discussed 
the  role  of  extremum  principles  in  computer  vision  (Mradv  and  Horn  (IV83J 
present  a  fuller  discussion),  and  criticized  Harrow  and  1  cnenh.ium  s  measure  on 
the  grounds  that  it  is  highlv  sensitive  to  noise.  In  Sections  4  and  5  we  analyzed 
the  extremum  principle  and  showed  that  it  corresponded  closely  to  human 
perception  of  oriented  planar  contours,  and  that  it  interprets  skew  symmetries 
as  oriented  real  symmetries. 

In  this  Section  we  return  to  the  discussion  in  Section  3.  concentrating  on  two 
additional  questions  concerning  our  measure.  First,  we  ask  whether  there  are 
specific  reasons  why  a  visual  system  should  adopt  the  sirateg)  of  extremizing 
our  measure.  We  suggest  that  die  extremum  principle  not  only  detennines  the 
orientation  of  the  viewed  curve,  but  provides  an  estimate  of  the  stability  of  the 
interpretation.  We  rcl.itc  this  to  the  idea  of  general  viewpoint.  Second,  we 
show  that  the  compactness  measure  is  rcladvcly  insensitive  to  noise  and  to  the 
scale  at  which  die  image  is  s.iinplcd.  lo  this  end  we  consider  two  cases:  (i) 
adding  a  sinusoid  to  the  contour,  (ii)  assuming  that  the  image  can  be  modelled 
in  terms  of  fr.ictals  (Mandelbrot,  1982). 

Mewing  position  and  stability 

Consider  viewing  a  given  planar  curve  from  the  hemisphere  of  all  possible 
directions.  Consider  further  the  way  the  image  changes  when  die  viewpoint 
is  shifted  slightly.  A  sminnh  curve  will  hardly  change  with  a  slight  change  of 
viewpoint  from  most  viewing  directions.  We  call  these  viewing  directions  stable 
viewpoints.  Stable  viewpoints  can  be  grouped  into  regions  whose  boundaries 
correspond  to  viewpoints  where  die  contour  changes  rapidly  for  a  slight  change 
of  viewing  position.  These  stable  regions  will  be  quite  large  for  images  of 
smooth  plan.ir  curves  and  smooth  curved  surfaces.  We  suggest  that  image 
contours  arc  mterpretated  as  curves  diat  are  viewed  from  stable  viewpoints. 
This  is  the  essence  of  die  fienrrnl  v/w/iw/i/ constraint  in  computer  vision.  We 
arc  able  to  estimate  how  stable  a  given  viewpoint  is. 

One  way  to  find  stable  viewing  positions  is  to  define  a  similarity  measure  for 
viewpoints  and  then  iind  the  extrema  of  diis  measure.  Sutheiland  [personal 
communiciiiionl  has  proposed  A/P^  as  a  similarity  measure,  b.ised  on  empirical 
studies  of  animal  perception,  if  this  is  correct,  extremizing  A/P^  corresponds 
to  finding  the  stable  viewing  positions  of  a  contour,  and,  as  a  result,  the  stable 
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iiucrprcliilions  of  a  coiuour.  defined  lo  be  those  vvhieli  obsei\ers  at  general 
positions  arc  most  likely  to  sec.  I'his  suggests  that  assuniing  general  \iewmg 
position  torresponds  to  assuming  interpretations  around  which  our  compactness 
measure  is  extremi/ed  and  where  it  varies  slowly. 

We  illustrate  lliis  idea  by  considering  an  ellipse.  1-or  an  ellipse  the  stable 
viewing  position  corresponds  lo  interpreting  die  ellipse  as  an  oriented  circle, 
hven  if  the  circle  is  slanted  by  as  much  as  thirty  degrees,  llic  value  of 
hardly  changes,  as  our  algorithm  demonstrates,  and  so  the  circle  is  a  stable 
intcrjirctation.  If  the  algorithm  is  applied  to  an  ellipse  with  large  eccentricity, 
corresponding  to  large  slant.  AjP^  changes  rapidly  for  slight  changes  to  tlic 
viewing  position.  Sty  if  we  look  at  an  ellipse  fiom  all  possible  viewing  angles 
it  is  most  likely  to  appear  as  a  circle  and  so  interpreting  an  ellipse  as  a  slanted 
circle  is  a  good  strategy. 

lliese  results  are  preliminary,  and  they  will  be  rigorously  developed  in  a 
further  paper. 

Sensitivity  of  the  compactness  measure  to  noise 

"llic  second  question  concerns  the  sensitivity  of  the  measure  to  noise  and  to 
the  scale  at  which  an  image  is  sampled.  Since  our  measure  involves  fewer 
derivatives  of  the  contour  than  measures  such  as  that  proposed  by  Barrow 
and  Tenenbaum.  it  should  be  relatively  insensitive  to  noise.  Although  our 
present  algorithm  and  tlic  sampling  metliod  arc  essentially  equally  sensitive,  it 
is  possible  to  develop  a  less  sensitive  algorithm  to  implement  our  measure,  nie 
sampling  method  inevitably  involves  calculating  tangents,  however,  and  hence 
is  inherently  sensitive  to  noise  and  scale,  l-'or  example  if  the  image  contour 
is  continuous,  but  not  dill'ercniiable.  tlic  sampling  mciliod  cannot  (strictly)  be 
applied.  Similarly  if  the  orientation  of  tlic  tangent  to  the  contour  has  a  large 
high  frequency  component,  the  sampling  approach  will  be  very  sensitive  to 
noise. 

Consider  the  sensitivity  to  noise  of  the  extremum  principle.  Clearly,  the 
perimeter  P  depends  on  the  scale  at  which  the  image  is  viewed,  while  the  area 
A  is  much  less  dependent.  It  follows  that  the  ratio  AjP^-  varies  with  scale. 
If  a  sinusoid  is  added  to  a  smooth  contour,  for  ex.implc.  the  area  will  remain 
approximately  the  same  but  the  perimeter  will  change  significantly.  Ihis  docs 
not  imply,  however,  that  the  extrema  o\'  A j P"  vary  with  scale,  l-'or  example,  it 
is  easy  to  show  that  if  a  sinusoid  is  added  to  the  contour,  the  extremum  of  the 
area  over  the  perimeter  squared  is  elfectively  unchanged. 

Simil.irly.  suppose  that  the  image  can  i.e  ajiproximaied  b>  a  fractal  (Mandelbrot. 
1982|.  If  we  ineiisure  the  image  at  a  sm.ill  fractal  scale  constant  /.  the  perimeter 
will  be  given  by 
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P(/)  = (b.l) 

whore  F  is  mdopcitdcni  of  /.  and  d  >  1  is  iho  fraclal  (iimcnsi<in.  Pnuided  I  is 
small,  the  \ahic  of  ,4  is  essentially  inde|)Cndetit  of  /.  siiilo  the  fr.ntal  tends  to  a 
limit  eonuuir.  with  finite  area,  as  i  goes  to  zero.  Now  we  cxircini/e  A/P^  for 
all  contours  which  can  be  projected  into  the  image  contour.  This  is  equivalent 
to  extremi/ing  log{v4/P^)  which,  using  (6.1),  we  write  as 

=  log  A  —  2  log  P  —  2(1  —  d)log/.  (6.2) 

We  now  extremi/o  this  (>\cr  o  and  t  (Hqs.  4.24  and  4.25),  and  note  that  the 
term  involving  I  is  independent  of  a  and  r  and  disappear.  It  follows  that  the 
extrema  of  Aj P^  are  independent  of  the  scale  of  viewing  for  an  image  that 
can  be  approximated  by  a  fractal. 

7.  Interpreting  image  contours  as  curved  surfaces 

Figure  3  shows  a  number  of  contours  th.it  arc  interpreted  as  curved  surfaces. 
In  this  section  we  discuss  one  method  for  extending  our  extremum  principle 
to  tliis  general  case.  The  key  observation,  as  it  was  for  W'itkin  (lOSl).  is  that 
our  method  can  he  applied  liKally.  lo  do  this,  we  assume  that  the  surface  is 
locally  planar.  At  the  surface  houiuKiry.  corresponding  to  the  deprojection  of 
the  inia.ce  contour,  the  binomial  coincides  with  the  surface  normal.  1110  idea 
is  to  compute  a  loc.il  estimate  of  the  surface  nonnal  by  ilie  extremum  principle 
described  in  the  previous  sections  and  then  to  use  an  algorithm,  such  as  that 
developed  by  Icr/opoulos  |198.3].  to  interpolate  the  surface  orientation  in  the 
interior  of  the  surtacc.  I  he  method  is  closely  rcl.ited  to  that  proposed  by  Brady 
and  Cirimson  (19811  tiir  perceiving  subjective  surfaces. 

I'hc  main  question  concerns  how  to  apply  tlic  extremum  principle  locally. 
We  are  currently  investigating  the  following  approach.  Consider  the  circle  of 
curvature  to  the  sp.icc  curve  that  is  the  deprojection  of  tlic  contour.  One  way 
to  define  the  circle  of  curvature  is  as  die  best  fitting  local  circle  through  three 
points  (or  more  if  one  assumes  noisy  data  and  makes  a  least  squares  estimate) 
on  the  space  curve  near  the  point  in  question.  I'he  circle  of  curvature  projects 
into  an  ellipse.  We  compute  tlic  best  fitting  ellipse  at  each  point  on  live  image 
contour,  and  compute  frr'm  it  a  local  estimate  of  the  surface  orientation  by 
finding  the  slant  and  till  of  tlic  corresponding  circle,  interpreted  as  the  circle 
of  curvature,  it  is  easy  to  show  that  so  long  as  the  surface  foreshortening  is 
differentiable,  this  is  a  good  estinnitor  of  the  circle  of  curvature  and  hence  of 
the  loc.il  surface  nonnal. 

It  requires  5  (i.ir.imeters  to  define  an  arbitrary  ellipse.  Computing  the 
best  fitting  ellipse  in  the  general  case  is  a  complex  nonlinear  problem  best 
,ip|iro.ii.hed  using  .i  numerical  dc"w.ciit  method,  though  several  .ilgoriilims  have 


20 


been  published  recenllv  fur  compuline  best  (iliini.’  ellipses  IHookstcin  1970.  Auin 
1981.  Nakai!a\»a  and  Roscnt'eld  1979.  Sampson  1982).  If  we  assume  ihal  ihc 
normal  to  the  space  eune  Is  not  signilicantly  loreshortened.  we  can  compute 
the  ellipse  center  and  major  axis  from  the  curvature  of  tlie  contour  at  tlie  point 
in  question.  The  slope  of  the  contour  at  that  point  also  dclincs  the  orientation 
of  tlie  ellipse,  leaving  a  much  simpler  one  p.iramctcr  problem. 

We  note  that  perccptuallv  the  strongest  local  cues  to  surface  orientation 
correspond  to  points  of  high  curvature.  I  his  is  consistent  with  our  method  of 
locally  estimating  surface  orientation  by  luting  local  ellipses..  Recall  that  our 
compactness  measure  was  inspired  by  the  observation,  for  example  on  ellipses, 
that  large  slant  is  an  cfl'cctive  cue  to  surface  orientation  and  in  the  case  of 
an  ellipse  tliis  produces  points  of  high  curvature.  In  fact,  we  can  show  that 
numerical  conditioning  of  the  estimator  of  sl.mt  increases  monotonically  with 
the  slant.  Conversely,  for  straight  line  portions  of  a  contour,  the  curvature 
is  zero,  and  the  surface  is  UKally  planar.  Hence  surface  orientation  docs  not 
change  along  the  length  of  the  straight  portion. 

One  issue  tliat  remains  to  be  studied  is  the  interface  to  the  surface  reconstruc¬ 
tion  algorithm.  Consider  the  image  of  a  triangle  shown  in  Figure  1.  Ihcre 
are.  in  general,  tlircc  different  perceived  orientations  of  tlic  tri.inglc  correspond¬ 
ing  to  propagating  the  interpretation  of  each  of  the  (high  curvature)  corners. 
Adjacent  corners  give  inconsistent  information,  and  so  it  seems  necessary  to  use 
a  labelling  appro.ich  such  as  Utat  proposed  by  /ticker.  Hummel,  and  Roscnfcid 
(19771. 

7.  Related  work 

Ciibson  [1 950)  has  argued  that  surface  orientation  is  directly  detennined  by 
certain  "higher  order  variables"  in  the  proximal  stimulus  array.  What  exactly 
constitutes  a  "higher  order  variable"  has  been  the  subject  of  extensive  debate. 
Nevertheless.  Gibson  and  his  followers  have  proposed  several  such,  especially 
for  optical  flow  and  texture  gr.idicnts.  flock  (1964.  fliK'k  ct.  al.  19^7)  have 
adopted  a  Gibsonian  perspective  on  the  judgement  of  slant  in  texture  gradients. 
He  has  introduced  a  "higher  order  v.iri.ible"  opucol  shtni  (actually  "optical 
theta"  in  tlie  original)  as  a  possible  basis  for  monocular  sl.mt  perception  (f  lock 
19fi4.  Iq.  .‘i)  or.  at  least,  as  a  discriminant  for  planarity,  flock  s  work  implicitly 
assumes  phin.irily  (f  lock  1964,  p.  .18l(.  and  it  assumes  th.ii  the  tilt  direction 
has  been  computed  previously.  In  fact,  there  arc  many  asMimpiions  in  f'hKk’s 
paper  (frecm.in  (1965.  p.  502)  counts  l.H.  Moreover,  seicial  studies  (Clark 
ft.  al.  1956,  Cmibcr  and  Cl.iik  1956.  frecm.in  1965.  Ilr.iunsicin  and  Payne 
1969)  have  shown  lh.it  the  dclcrmin.nion  of  surface  orient. ilion  from  texture 
gr.idicnts.  for  example  using  optical  slant  or  as  in  Ikeuchi  (198.1).  is  less  clfcxtivc 
til. in  its  dctcrmiii.ition  lioin  bounding  contour. 
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(a) 


(b) 


Figure  7 

a.  A  box-like  figure  typical  of  those  studied  by  Attneave  and  Frost  |1%9).  b.  A  sheared 
box. 


Attneave  (1972.  see  also  Attneave  and  Frost  1969)  describes  an  approach 
to  detemiining  stnfacc  orientation  that  has  similarities  to  that  developed  here. 
Fitst,  he  argues  for  a  Praegnanz  theory  of  perception,  which  sircs.scs  economy 
principles  in  perception.  I  his  has  meant  diirerem  things  to  different  researchers. 
The  Gcsutlt  psychtilogists  explicitly  noted  the  link  between  Praegnanz  and 
minimization  principles  (for  example  the  soap  bubble)  in  mathematical  physics. 
Attneave  (1972.  p.  28.5)  suggests  that  such  minima  may  be  computed  by  "hill- 
climbing  techniques".  Hie  extremum  principles  discussed  in  Section  3  of  this 
paper  and  surveyed  in  (Brady  and  Horn  1983,  Grimson  1981.  1982,  1983,  and 
l  erzopoulos  1983)  can  be  considered  to  be  more  sophisticated  formalizations  of 
similar  intuitions. 

In  fact,  Attneave  adopts  Hochberg's  formulation  of  tlie  Praegnanz  theory  as 
the  tendency  ui  keep  didcrenccs  to  a  minimum.  In  particular  he  considers 
three-dimensional  interpretations  that  equalize  one  or  more  of  angles,  lengths 
of  edges,  and  surlace  slopes  in  figures  such  as  Figure  7a.  The  more  of  these 
tliat  are  in  fact  equalized  in  a  particular  three-dimensional  interpretation  of  an 
image,  the  more  likely  that  interpretation  is  to  be  chosen.  In  fact.  tlic  extremum 
principle  developed  in  this  paper  will  interpret  Figure  7a  correctly.  It  will  also 
determine  the  shear  in  Figure  7b. 

Second.  Attneave  (1972.  Fig.  4)  considers  the  judged  orientation  of  rhombus 
figures  (see  Appendix  I)  below  for  analysis  of  this  case  by  our  extremum 
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principle.)  Surface  uriciualion  can  be  judged  reliably  for  such  shapes,  mure 
reliably  in  fact  than  for  the  box  figures  shown  in  Kigure  7.  so  long  as  the 
suiiinetrics  of  the  rhombus  do  not  align  with  the  hori/onlal  .md  \crtical.  It  is 
well  known  that  the  hori/ontal  and  vertical  arc  important  for  shape  description, 
(see  for  example  Ifrady  |19831  for  discussion).  Note  that  suifacc  slant  is 
consistently  underestimated. 

In  order  to  express  depth  constraints.  Barnard  11982)  and  Ikcuchi  |1983J 
have  proposed  alternative  projections  to  orthographic  projection  used  in  the 
development  in  Sections  3  and  4.  Ihc  mctliods  we  have  developed  extend 
straightforwardly  to  tlicsc  alternative  projections. 

Barnard  (1982)  finds  surface  orientation  from  vanishing  points  of  families  of 
parallel  edges  using  central  or  perspective  projection.  He  considers  the  projec¬ 
tion  of  an  angle,  and.  after  the  fashion  of  Hochberg  and  Attncavc.  heuristically 
assumes  equi-angularity  to  arrive  at  a  three-dimensional  interpretation  of  a 
triangle.  As  noted  in  Appendix  C.  our  method  dispenses  with  the  need  for  such 
a  heuristic  assumption.  Similar  remarks  apply  to  Barnard's  study  of  curvature. 

Ikcuchi  )1983)  proposes  a  method  for  determining  surface  orientation  for  a 
surface  that  is  covered  with  uniformly  repeating  texture  elements.  He  assumes 
that  the  surface  element  corresponding  to  tlic  texture  element  is  planar.  Most 
critically  it  is  assumed  that  tlic  shape  of  the  texture  element  is  known  a  priori. 
l-'or  a  particular  slant  and  tilt,  the  orientation  of  the  projection  of  the  known 
figure  changes.  Ikcuchi  proposes  a  measure  tliat  is  stipcrficially  similar  to  the 
symmetry  measure  in  Section  4  to  determine  tlic  slant  and  tilt  of  a  texture 
element 

Olson  (1974)  has  studied  a  version  of  Ames'  trapcroidal  window  illusion. 
Surface  slant  is  judged  more  accurately  when  die  stimulus  is  moving  consistent 
with  the  static  three-dimensional  interpretation  than  in  die  purely  static  case. 
Similarly.  Wallach,  Weis/,  and  Adams  |1956]  have  shown  that  when  an  ellipse 
is  rotated  in  die  image  plane  about  an  axis  passing  through  its  center  and 
noimal  to  the  image  it  is  perceived  as  a  spinning  oriented  circle,  like  a  settling 
spinning  penny.  I  hc  instantaneous  surface  orientation  of  the  oriented  circle 
can  be  computed  by  vmr  meibod.  Hildreth's  forthcoming  Bhl)  diesis  on  the 
perception  of  motion  discusses  the  rotating  ellipse  and  proposed  a  dieorctical 
explaii.ition  of  human  perception  of  it. 
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Appendix  A;  i  lie  nuixiimnn  likeliintod  inelliod  applied  to  an  ellipse 

Hquaiidii  (4.30)  gi\cs  the  value  F  of  titc  logarithm  of  the  Maximum  l  ikelihood 
estimator  for  surface  orientation: 

F  =  log  cos  a  j'  dr  —  ^  log(cos^  a  -f  sin*  ac.os^{a  —  ■r))dr.  (^-1) 

In  this  Appendix  we  investigate  the  slant  computed  by  the  Maximum  Likelihood 
method  for  the  ellipse 


I*  y* 

- L  —  =  1. 

a2  ^  62 


It  is  convenient  to  use  the  standard  parameterization. 


{A.2) 


X  =  a  cos  6 

y  =  bsm  6,  0  <  $  <  2n 

dr  =  (a*  sin*  5  +  6*  cos*  9)  ^  dO 

Pending  a  more  thorough  analysis,  we  assume  that  the  tilt  t  —  it/2  is  computed 
correctly  by  the  Maximum  l.ikclih<H)d  method,  and  restrict  attention  to  slant. 
Witkin  |I981.  l-'igure  5|  and  our  own  computational  experiments  suggests  that 
this  is  reasonable:  in  any  case,  if  it  is  not  so,  the  inequivalence  of  the  Maximum 
Likelihood  and  Hxiremurn  Principle  methods  would  be  assured.  Without  loss 
of  generality,  we  assume  tliat  a  >  b.  and  denote  a/b  by  X  >  1. 

Since  r  =  ■jr/2. 


COS*Q 


1 

1  -f-  tan*  a 
0*  sin*  9 

o*  sin*  0  -f-  6*  cos*  9 


since  tan  a  =  dyfdx.  Also. 


(AA) 


8in(Q  —  r)  =  sin(Q  —  -)  = 

A 


—  COSQ, 


(A.5) 


and  so  I’q.  (A.l)  reduces  to 


F  =  log  cos  CT  ^  dr  —  ^  log(  1  —  cos*  a  sin*  a)dr. 


(A.6) 


W'c  arc  interested  in  extrema  of  F.  and  so  we  consider  dFfda,  which  we 
write 
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dF  /  sin^  ^  cos^  o  —  cos^  ^ 

:o  -  =  f - - dr,  1  <  X 

■'  sin^  cos^  o -j"  0 


Suppose  that  the  Maximum  l  ikelihood  method  is  extrcmi/cd  at  a  =  a'. 
The  slant  that  is  necessary  to  interpret  tlte  ellipse  as  a  circle  is  given  by 
COSO  =  1/X.  The  Maximum  I  ikeliJi(«>d  method  overestimates  slant  if  and 
only  if  o'  >  c<)s~'*(l/X).  which  is  if  and  only  if  cose'  <  1/X.  This  is  true 
if  dFjda  ^  0  for  all  a  such  that  1  <  Xcosa  <  X.  Denote  Xcosa  by  /x, 
so  tliat  1  <  /X  <  X.  Substituting  in  Fcj.  (A. 7)  and  changing  tlic  limits  of 
integration,  we  find 


dF  /x^  sin^  —  cos^  ^ 

-  =  2o  / - r - 

da  Jo  ^2  sjjjX  Q  _|_  j,Qg2  Q 


(X^sin^  9  +  cos^ 


d)U9 


(yl.8) 


We  now  split  the  range  of  integration  into  four  equal  intervals  of  si/c  ir/4. 
With  suitable  changes  of  variables  to  bring  the  intervals  of  integration  to  [0,  J], 
we  find 


dF^ 

da 


a^cos^B  —  sin^S,,  5  5^  .  2 

2  27-,  •  ^ 

/x^  cos^  6  sin^  9 


Ij}  sin^5  —  cos^  9 
/x^  sin*  9  +  cos*  9 


(X*  sin*  9  +  cos*  9)^^d9 


{A.9) 


Over  the  range  of  integration.  cos0  >  sin  9.  Algebraic  manipulation  of  Rq. 
(A.9)  shows  it  to  be  greater  than  rcro,  so  tliat  it  is  certainly  not  zero.  Hence 
the  Maximum  Likelihood  method  overestimates  Uie  slant. 
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Appendix  I):  \\  h>  square  curxuture  is  inappropriate 
In  section  3  wc  rejected  the  measure 

F  =  J  K^ds  (/i.l) 

because,  despite  popular  belief  to  the  contrary,  it  docs  not  interpret  an  ellipse 
as  a  circle.  In  lliis  appendix  we  substantiate  that  claim. 

Consider  the  ellipse  delincd  by  Hqs.  (A.2)  and  (A. 3).  The  curvature  k  of  the 
ellipse  is  given  by 


^2  = 


ds^ 


(a^  sin^  6  -F  b'^  cos^  5)® 


(R2) 

(H.3) 


where  s  is  the  arc  length.  Substituting  Fquation  (B.3)  into  (H.l)  w'e  have 


_ _ 

(a^  sin^  6  b^  cos^  0)^ 


d9 


(Z3.4) 


Suppose,  without  loss  of  generality,  tliat  the  ellipse  is  in  die  image  plane  and 
that  a  >  b.  Stipposc  further  that  the  square  curvature  performance  index  (B.l) 
is  correct  and  interprets  tfic  ellipse  as  a  circle  lying  in  a  plane  that  is  slanted 
with  angle  a  to  die  y-axis,  w'herc 


b=OCOS(7 

Fquivaicntly.  if  we  set  b'  =  b/cosa.  the  measure  will  choose  o  such  diat 
b'  —  a.  Hence  if  we  write  b  =  Xo.  and  consider  F(a,b)  to  be  a  function  of 
X,  die  measure  should  be  extrcmi/cd  by  X  =  1.  Wc  will  now  show  that  Uiis  is 
no!  the  case. 


1 

r-’(X)  =  - 

a  Jo 


(sin^  -F  X^  cos*  9)^ 
Ditfcrcntiating  this  with  respect  to  X  gives 

1  2X  sin^  ^  —  3X®  cos*  0 


de 


OF 

d\ 


=  ■/ 
a  Jo 


®  (sin*  5  +  X*  cos* 


d9 


(13.6) 


(B.l) 
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hvalu.iting  tliis  expression  at  X  =  1  gives 


dF 

ax 


x=i 


1 

-  I  (2sin^^  —  3cos^0)(i0 

a  JO 
IT 


a 


(B.8) 


Since  the  partial  of  F  with  respect  to  X  docs  not  vanish  when  X  is  equal  to 
one,  tlic  circle  docs  not  extremize  the  ellipse  for  the  square  curvature  measure. 
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Appendix  C:  I  he  iiiterprelntion  of  sonic  simple  shnpcs 

In  this  appendix  wc  show  that  our  method  correctly  interprets  a  number  of 
simple  shapes,  namely  an  ellipse,  a  parallelogram,  and  a  triangle. 

I'llipse 

Suppose  the  ellipse  is  gixen  by  equations  (A.2)  and  (A.3).  It  is  easy  to  show 
tliat  ilic  area  A  and  perimeter  P  arc  given  by 

A  =  irab 

P  =  (a^  s\n^  9 cos^  0)^ d6 

Maximizing  A/P'  is  equivalent  to  minimizing  P/\/A.  Wc  set  b  =  Xa,  (X  is 
die  eccentricity  of  the  ellipse,)  and  define  P/\/A  =  /(X).  Wc  find 

/(X)=—  /  +  Xcos2d)i(ifl  (C.2) 

Ry  the  same  argument  presented  in  appendix  B,  the  ellipse  will  be  interpreted 
as  a  circle  provided  X  =  I  is  a  minimum  of  /(X).  Changing  the  variable  of 
the  integral  to  0  =  ^  -j-  |  wc  find  that 

/{X)  =  /(i)  (C.3) 

which  implies  that  X  =  1  extremizes  /(X).  It  is  also  clear  that  extrema  occur 
in  pairs  of  the  form  X,!/X.  l-uvihcrmorc.  both  arc  stationary  points  or  one 
is  a  maximum  and  the  other  a  minimum.  Observe  that  /(X)  tends  to  infinity 
as  X  tends  eitJicr  to  zero  or  infinity.  It  follows  tliat  a  sufficient  condition  for 
X  =  1  to  be  a  global  minimum  is  that  all  pairs  of  extrema  be  suilionary  points. 
Suppose  that  this  is  not  the  case,  and  let  Xq  be  die  smallest  extremum  that  is 
not  a  suuionary  point.  Since  /(X)  tends  to  infinity  as  X  tends  to  zero,  Xq  must 
be  a  minimum  and  I/Xq  a  maximum.  But  I/Xq  is  the  largest  non-stationary 
extremum,  and  so.  by  the  same  argument  as  above,  it  must  be  a  minimum. 
This  contradiciion  establishes  the  result 
PonilUwyjain  (iml  triangle 

In  section  5  wc  showed  that  a  skew  symmetry  is  always  interpreted  as  an 
oriented  svnimetry  by  our  method.  In  particular,  a  parallelogram  is  interpreted 
as  a  rectangle.  By  the  siimc  argument,  a  rectangle  is  a  skewed  symmetry  of  a 
square.  Hence  our  method  interprets  a  paralleltigram  as  a  square. 

Similar  reasoning  shows  that  a  triangle  is  interpreted  as  a  skewed  isosceles 
tri.inglc,  which  is  intcnirctcd  as  a  skewed  equilateral  triangle.  Hie  axes  of  the 
skewed  symmetry  join  a  vertex  to  the  midpoint  of  the  opposite  side.  Hence 
our  method  interprets  a  triangle  as  an  oriented  equilateral  triangle. 
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Appendix  I):  I'he  slant  and  tilt  of  a  skened  symmetry 

In  this  appendix  we  calculate  the  slant  o  and  the  till  t  that  correspond  to  the 
oriented  real  symmetry  that  is  the  interpretation  of  a  skewed  symmetry  whose 
skew  angle  is  6. 

As  a  simple,  though  instructive,  example,  consider  a  rhombus  of  side  a  and 
included  angle  7  (Figure  8).  lo  find  the  extremi/ing  tilt,  we  subsiituic  ilie  dau 
from  Figure  8  into  Fq.  4.24,  and  find 

sin  r  COST  sinfr  —  7)cosfr  — 

- -  4. - ' - LL_  =  0  (D.l) 

|l  —  sin^trsin^rj  -  sin^t7sin2(T  -  7)| 

We  can  rewrite  this  in  the  form 

|cos^(t  —  7)  —  cos^  r|  |cos^(r  —  7)  +  cos^  r 

+  (1  —  cos^a)(l  —  cos*(t  —  7))(1  —  cos^t)  —  1 
=  0 

(D.2) 

We  assume  first  that  the  first  factor  is  /ero.  It  follows  that 


and  so 


co3(t  —  7)  =  ±  cos  T 


(D.3) 


n  ~2t  ~  nn.  [DA) 

Since  0  <  r  <  tt  and  0  <  7  <  tt.  there  arc  two  possible  solutions,  namely 


T  _  2 

2 

_  7+  tt 
2 


[D.b) 

{D.S) 


Observe  that  the  tilt  direction  is  one  of  the  axes  of  symmetry  of  the  rhombus 
slmwn  in  Figure  8. 

Having  solved  for  the  tilt,  we  now  solve  for  the  extremi/ing  slant  o  using 
Fq.  4.2.V  Keealling  from  Fq.  I).4  that  sin^(T  —  7)  —  .sin^  r.  we  find  upon 
substitution  into  F.q.  4.23 


-jl  —  sin^  o  sin^  r|  =cos*t|i 


•  2  -21 

sin  a  sin  t] 


(D.7) 
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which  wc  solve  lo  get 


2  cos^  T 
cos  (J  = - . 

sin^  T 


(D.8) 


'Hie  requirement  that  |cosa|  <  1  picks  out  either  l-quati('n  (I).5)  or  (I).6). 
so  we  get  a  unique  solution.  In  tliis  case  the  skew  angle  is  given  by 


6=^-7.  (D.9) 

To  summari/c.  if  <5  <  0  we  get 


and  if  ^  >  0, 


T  = 

COSO  = 


•n  6 

4~  2 
1  +  sin  i 

costf  ’ 


(D.IO) 


COSO  = 


^  _6_ 

T  ~  2 

cos^ 

1  -t-  sin  6  ’ 


(D.ll) 


'lltesc  formulae  were  derived  for  an  equal-sided  parallelogram  but  they  will 
clearly  apply  to  the  more  general  case  and  a  rotation  through  the  angles  given 
by  f-'quations  (1)10)  and  (l).ll)  will  unskew  any  symmetry.  It  should  be  noted 
that  T  is  taken  to  be  zero  on  the  axis  of  synvmcUy.  as  in  Figvire  S. 

lo  conclude  this  Appendix,  we  consider  the  case  that  the  second  factor  in 
Kq.  (I).2)  is  zero  and  the  first  factor  non-zero.  We  introduce  the  angle  rp  by 
analogy  with  Iq.  (2.1)  defining  orthographic  projection: 


Then 


tan*  V*  =  cos*  o  tan*('7  —  t) 


P-12) 


sin*(7  —  r)  = 


cos*(7  —  r)  = 


tan*  t/> 

cos*  o  -f-  tan*  ip 
cos*  o 

cos*  o  -f-  tan*  ip 


(Z).13) 


where,  without  loss  of  generality,  we  suppose  0  <  ip  <  7r/2.  Now  by 
assumption,  the  second  factor  in  Hq.  (I).2)  is  zero: 


cos*  T  -|-  co.s*(7  —  r)  -F  sin*  ersin*  rsin*(7  —  t)  —  1  =  0  (^-14) 
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Figure  8 

A  typical  skew  symmetric  figure,  namely  a  rhombus  of  side  a  and  included  angle  7.  with 
skew  angle  <5  =  1-7.  The  tilt  direction  is  aligned  with  one  of  the  :aes  of  symmetry 
of  the  rhombus,  determined  by  the  angle  S. 


Using  hlq.  (D.13),  wc  find 


from  which  wc  deduce 


tan^Ttan^  V*  =  U 


sin  r  =  cos  rj) 

cosr  = /.t  sin  =  iU 

since  the  ranges  of  liic  variables  can  be  asstimed  to  be  0  <  a  < 
r  <  7r;0  <  r/)  <  7r/2;0  <  7  <  TT.  From  I'q.  (13.13)  wc  find  that 


(D.IS) 
n/2;0  < 


sin(7  —  r)  cos(7  —  r)  = 


u  tan  V  cos  a 


±1, 


(D.16) 


cos^  a  -)-  Ian*  t/*  ’ 

and,  since  sin  r  cos  t  has  the  stime  sign  as  sin(7  —  t)cos(7  —  t).  h  =  u.  From 
F,q.  (l).l)  wc  deduce 


sin  T cost  —  sin(7  —  r) cos(7  —  r) 

1  —  sin*  a  sin*(7  ~ 
t'sin  ipcosil) 

1  —  sin*  0  cos* 


1  —  sin* a  sin* 


(0.17) 
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So  far  in  this  Appendix  we  have  only  used  one  of  the  constraints  derived  in 
Section  4.  namely  Hq.  {4.241  (from  which  we  derived  Hq.  (D.l)).  We  now  use 
the  second  constraint  liq.  (4.23).  which  we  can  write  in  the  form 


2  cos*  T 

(1  —  sin*r;8in*r)i 


—  (1  —  sin*  a  2r)i 

2  co8*(7  —  t) 

(1  —  sin*  a  8in*(7  —  r))i 

—  (1  —  sin*  o  8in*{7  —  t))^  =  0 


(£1.18) 


After  some  algebraic  manipulation,  we  deduce 


COSO  =  —  tan*  rp, 

from  which  it  follows  that  cos  a  is  negative,  which  is  impossible  in  the  range 
under  consideration. 
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